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Background / Context: A substantial literature documents the benefits of early childhood 
education and formal preschool experiences on children’s school readiness, with low-income and 
otherwise disadvantaged children benefitting the most from these programs (Yoshikawa et ah, 
2013). However, these academic benefits often fade out as children age, and most disappear by 
the end of kindergarten or first grade (Barnett, 1995; Currie, 2001; Puma, Bell, Cook, & Heid, 
2010). A meta-analytic study estimated the magnitude of preschool intervention fadeout at .025 
standard deviations per post-treatment year (Leak et al., 2013). 

Little research has focused on why short-tenn gains from preschool may disappear and 
the conditions under which gains from preschool might be sustained into elementary school 
(Claessens, Engel, & Curran, 2013). One hypothesis of preschool fade out is that children’s 
elementary school teachers continue to teach content that children already learned during 
preschool, thus curtailing academic growth. Indeed, recent work suggests that spending too 
much instructional time on content already mastered by students may temper achievement gains, 
whereas exposure to more advanced content in kindergarten could bolster new skill development 
(Engel, Claessens, & Finch, 2013; Magnuson, Ruhm, & Waldfogel, 2007). 

In the current study, we investigate two salient approaches available to policymakers that 
may improve preschool participants’ instructional experiences in elementary school. The first 
involves advanced and challenging instruction in kindergarten and first grade, because children 
who attend preschool will hypothetically benefit more from advanced content. The other 
involves some type of professional support in which preschool teachers interact with their 
kindergarten and first grade counterparts to develop a seamless transition from one grade to the 
next. We use two experimental studies of preschool interventions and children’s elementary 
school environments to examine whether the quality of instructional content or providing 
professional development supports to early grade teachers moderate the impacts of two well- 
known programs on children’s cognitive skills: Head Start and Building Blocks. 

Research Questions: 

1 . Does the quality of academic instruction in kindergarten and first grade moderate the 
magnitude of preschool intervention effects on children’s academic skills in kindergarten and 
first grade? 

2. Does a professional development intervention for kindergarten and first grade teachers that 
provided techniques designed to build upon the preschool program moderate preschool 
intervention effects on children’s academic skills in kindergarten and first grade? 

Setting and Intervention: This study is a secondary data analysis of data collected on children 
participating in two preschool interventions — Head Start and the Building Blocks preschool 
mathematics intervention. Head Start is a comprehensive child development program that 
provides children with preschool education, health screenings and examinations, and nutritious 
meals, in a full-day, center-based setting. The Head Start children in our sample participated in 
the program during their pre-kindergarten year at age 4 at different research sites across the 
country. The Head Start Impact Study (HSIS) evaluation began in 2002 (described below). 

Building Blocks (BB) is a preschool mathematics curriculum that encourages the 
acquisition of conceptual and procedural knowledge in both numeracy and geometric/spatial 
reasoning through the emphasis of empirically-supported learning trajectories (see Clements & 
Sarama, 2008). The TRIAD (Technology-enhanced, Research-based, Assessment, and 
professional Development) evaluation study was designed to assess the long-run impacts of BB 
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in 42 public elementary schools operating state preschool programs serving low-income 
communities in Boston, Massachusetts and Buffalo, New York. Study schools were assigned to 
one of three conditions: 1) BB preschool curriculum; 2) BB preschool curriculum with follow- 
through; 3) control (business as usual). Children in schools assigned to the two BB groups 
received the BB curriculum during preschool (age 4), and preschool teachers attended 13 study- 
administered pedagogical development (PD) sessions throughout the preschool year. Teachers in 
schools assigned to the “BB with follow-through” group received additional PD designed to help 
bridge the gaps between preschool, kindergarten, and first grade. These additional PD sessions 
brought teachers from all three grades together to discuss what students leam in each grade, and 
minimize the amount of repeated content. 

Population / Participants / Subjects and Data Collection: 

Head Start. The Head Start sample comes from the HSIS experiment dataset, which is a 
nationally representative sample of Head Start participants and a group of comparable non- 
participants. The full sample includes newly entering 3 -and 4-year old Head Start applicants who 
were randomly assigned to receive the Head Start program or a control group that did not enroll 
in Head Start where parents either found other available services for their child or the child was 
cared for at the home. Baseline survey and child assessment data were collected by study 
investigators (Westat) in the Fall of 2002, at post-treatment child assessments were collected at 
the end of Head Start in Spring 2003, and during kindergarten and first grade in Spring 2004 and 
2005. Our analyses use the 4-year-old cohort (n=1080) so that the children in both of HSIS and 
BB analyses received the preschool intervention during the same developmental period. 

The children and families in the sample are all very low income and have the following 
characteristics: 45% Hispanic, 39% white and 15% Black, 42% of parents have less than a high 
school degree, 23% are recent immigrants, 16% are teenage mothers and a majority (84%) live in 
an urban area. Infonnation on children’s elementary school experiences were collected from 
kindergarten and first grade teachers through a teacher survey in the spring of 2004 and 2005. 
Literacy skills were measured with the Peabody Picture Vocabulary Test (Dunn & Dunn, 1997) 
and Letter Word and Spelling standard scores from the Woodcock- Johnson Psycho-Educational 
Battery-Revised III (Woodcock, McGrew, & Mather, 2001). We created a literacy assessment 
composite measure to use as the dependent variable by standardizing all three measures to mean 
0 and standard deviation of 1, averaging across the three, and then restandardizing the measure. 

Building Blocks. TRIAD study participants (n=1375) were randomly selected from study 
schools at the beginning of the preschool year (2006-2007). The current study sample (n= 965) 
consists of students who had valid achievement data in preschool, kindergarten, and first grade, 
and at least one non-missing classroom observational measure in kindergarten or first grade. In 
the study sample, 35% of students were assigned to the BB group and 36% were assigned to the 
BB with follow-through group. The majority of students qualified for free or reduced price lunch 
(79%); 58% identify as African American and 20% as Hispanic. Math achievement was 
assessed at preschool entry, and at the end of the preschool, kindergarten and first grade year via 
the Research Based Early Mathematics Assessment (REMA; Clements, Sarama, & Liu, 2008). 

Research Design and Data Analysis: 

Variables of interest. We identified two key instructional characteristics in both studies to 
operationalize the quality of elementary school exposure to literacy content (HSIS) and math 
content (BB). In the HSIS, teachers were asked how many times in the past week their class did 
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a given literacy activity. We coded each activity into basic or advanced based on grade level 
(available in Appendix A). We converted each basic and advanced activity into times per month 
by taking the mean value of the answer category (e.g., Never=0, 1-2 times per week=1.5), 
multiplied by 4, and then standardized this measure to have a mean of 0 and standard deviation 
of 1 . Instructional quality during the first grade year is a cumulative measure of quality from 
both Kindergarten and first grade, averaged across the two years. 

In the TRIAD evaluation of BB, teachers’ instructional practices were evaluated via the 
Classroom Observation of Early Mathematics Environment and Teaching (COEMET; see 
Clements, Sarama, Spitler, Lange, & Wolfe, 2011). The COEMET is composed of 28 Likert- 
scaled items. Assessors, who were blind to treatment group, rated classrooms for teaching 
practices known to support early math development, such as the use of engaging small group 
activities and emphasizing cognitively demanding concepts and strategies. For the kindergarten 
year, we took the average of these 28 items and then standardized scores. As with the HSIS, our 
measure of first grade instructional quality is the standardized average of a child’s kindergarten 
and first grade COEMET scores. We also included the number of mathematical activities 
observed during each COEMET period in our analysis as an indicator of the amount of time 
spent on mathematics in the class. 

Analysis: We use multivariate regression to estimate the effect of instructional 
experiences on the magnitude of preschool treatment effects in children’s kindergarten and first 
grade year. Both preschool interventions were randomly assigned, so treatment effects estimated 
during the pre-k year and after are unbiased. In the HSIS models, we focus on literacy outcomes 
as the dependent variable, and in BB models, we focus on mathematics. In all models, we 
regressed achievement measures (taken at either end of preschool, kindergarten, or first grade) 
on treatment status, fixed effects for unit of random assignment, baseline assessment scores, and 
a set of control variables, varying slightly between BB and HSIS (see Tables 1 and 2 for details). 
We then add measures of classroom instruction as covariates to see how much of the treatment 
effect is explained by high-quality instructional practices. Finally, we add models in which 
treatment is interacted with classroom instruction. If high-quality instruction in kindergarten and 
first grade helps reduce fade-out, then these interactions should be positive and significant. 

Findings / Results: 

Table 1 presents the results for models estimating end of kindergarten effects, and Table 2 
presents end of first grade effects. All variables except the treatment indicator are scaled in 
standard deviation units to facilitate their interpretation as effect sizes. 

Kindergarten 

Head Start. Models 1-4 in Table 1 are for the HSIS where the dependent variable is a 
composite of three literacy and language assessments. Model 1 shows the end of Head Start year 
treatment effect, which is significant with an effect size of . 16. However, this effect becomes 
negative and insignificant by the end of Kindergarten (Model 2). When we add the instructional 
quality variables (basic and advanced literacy instruction) in model 3, the treatment effect 
remains unchanged though the coefficient on advanced literacy activities is .38 and significant, 
and the coefficient on basic literacy activities is -.28 and significant. Model 4 adds the 
interactions between instructional quality and treatment. Neither tenn was statistically 
significant, suggesting that advanced literacy instruction is unable to sustain the gains of the 
Head Start treatment group children through the kindergarten year. 

Building Blocks. In Table 1, Models 5-10 display the impacts of BB on kindergarten 
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mathematics achievement. Model 5 shows the BB treatment effect at the end of the preschool 
year with an effect size of .67. At Kindergarten (model 6) the effect drops to .34 and remains 
significant. When we add the instructional quality variables (COEMET and number of math 
activities) in model 7, the treatment effect remains unchanged, and the coefficient on the number 
of math activities is .13 and significant. Model 8 adds the interactions between instructional 
quality and treatment, but neither term was significant. 

Models 9 and 10 take a different look at sustaining pre-K gains. Rather than classroom 
quality, they examine a kindergarten teacher PD focused directly on sustaining the BB preschool 
gains. Unlike our measures of teacher-driven classroom instructional quality, teachers were 
randomly assigned to engage in additional PD. The treatment effect for students in the “BB with 
follow-through” was .37 and significant, but it was not significantly different from the end of 
kindergarten impact for students who received BB without follow-through PD (.33). All told, 
there is no evidence that either instructional quality or focused PD is able to sustain pre-K gains 
through the end of kindergarten. 

First Grade 

Head Start. Models 1-3 in Table 2 show the effect of Head Start on our literacy 
composite at the end of first grade. In each model, the treatment effect is not significant. 
Including the instructional quality variables does not change the significance of the coefficients, 
though adding the interaction tenns in model 3 increases the treatment coefficient to .11. Neither 
the instructional variables nor the interactions with treatment were significant. Again, there is no 
evidence that high-quality instruction can sustain Head Start gains through the first-grade year. 

Building Blocks. Models 4-7 show the effect of BB on math scores at the end of first 
grade. Models 4-6 show the BB only treatment effect at the end of first grade, which falls to . 17 
(from .33 in kindergarten) but remains significant. When we add the instructional quality 
variables in model 5, the BB only treatment effect falls from significance and there is a 
significant main effect of . 17 for the number of math activities. Adding the interaction with 
treatment in model 6 produces a similar pattern, where the BB only treatment effect is not 
significant at the .05 level, but there is a .22 significant effect size for the number of math 
activities. The interaction between instructional quality and treatment was not significant. Thus, 
yet another test shows no evidence that high-quality instruction sustains pre-K gains. 

As with the final models in Table 1, model 7 in Table 2 focuses on a PD approach to 
sustaining gains. A comparison of treatment effects for both the BB only and the BB plus follow- 
up group shows that the effect size for the follow-up group is .3 1 and significant, compared with 
a . 18 effect size for the BB only group. The .3 1 coefficient is only slightly smaller than the .37 
coefficient found on the PD group at the end of kindergarten, suggesting very little fadeout 
during first grade. In other words, two years of PD may be able to help sustain gains, although a 
comparison of the two effect sizes revealed that the follow-through group effect is not quite 
statistically significantly larger than the preschool only effect (F= 2.57; p = .11). 

Conclusions: We did not find any evidence to support the hypothesis that better instructional 
quality mitigates the fadeout of preschool treatment effects during elementary school. However, 
we did find some evidence that when the BB intervention was coupled with teacher professional 
supports in kindergarten and first grade, this all but eliminated the fadeout of effects observed 
between kindergarten and first grade. However, both the focused PD and high instructional 
quality could not reduce fadeout effects between preschool and kindergarten. 
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Appendix B. Tables and Figures 


Table 1. Head Start and Building Blocks treatment effects at the end of pre-k and Kindergarten 


Head Start Impact Study 
(Age 4 cohort) 

Building Blocks 

(1) (2) (3) (4) 

Literacy Composite Score 

(5) (6) (7) (8) (9) (10) 

Math Achievement 

End of End of Kindergarten 

Pre-k 

End of End of Kindergarten End of End of 

Pre-k Pre-k K 

Preschool Treatment 


Head Start Treatment Group 0.16* -0.14 -0.15 -0.11 

(0.08) (0.10) (0.10) (0.20) 

Building Blocks Preschool 
Treatment Only 

Building Blocks Preschool 
Treatment with K and 1st Grade PD 
Extension 

0.67** 0.34** 0.33** 0.33** 0.68** 0.33** 

(0.08) (0.07) (0.08) (0.08) (0.08) (0.07) 

N1 N1 N1 N1 0.63** 0.37** 

(0.10) (0.08) 

Instructional Quality 



Total advanced literacy activities in 

0.38** 

0.43* 

K (times per month; standardized) 

(0.13) 

(0.20) 

Total basic literacy activities in K 

-0.28* 

-0.29 

(times per month; standardized) 

(0.13) 

(0.25) 


COEMET (Good Instruction) 

0.04 

0.03 


(0.04) 

(0.06) 

Number of Math Activities 

0.13** 

0.14+ 


(0.05) 

(0.07) 


Treatment Interactions 


Head Start Treat * Advanced 
literacy activities 

Head Start Treat * Basic literacy 
activities 


- 0.10 

( 0 . 22 ) 

0.03 

(0.31) 


Building Blocks Treat * COEMET 


0.04 

(0.08) 


Building Blocks Treat * Number 
of Math Activities 


- 0.01 

(0.08) 


Controls 
Fixed effects for Random 
Assignment Block 

Observations 

**p<-01, * P< .05, + p< .10 
NI= Group not included in model. 


x 


x xx 


x x 


x 


x 


x 


x 


x x 

1624 1067 


x_ 

1067 


x_ 

1067 


x 

564 


x x 

564 555 


x 

555 


x x 

884 884 


HSIS Notes. Standard errors clustered by 
unit of random assignment (in 
parentheses). Controls: Head Start entry 
assessment scores, gender, race, teen 
mom, parents education, immigration 
status, parent age, urbanicity, treatment 
'no-show', treatment 'crossover', random 
assignment block. 


BB Notes. Standard errors clustered at the preschool teacher 
level (in parentheses). Controls: preschool entry math score, 
gender, race, age at preschool entry, mother's education level, 
free/reduced price lunch status, whether limited English 
proficient, random assignment block. 
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Table 2. Head Start and Building Blocks treatment effects at the end of First Grade 


Head Start Impact Study 
(Age 4 Cohort) 

Building Blocks 

(1) (2) (3) 

Literacy Composite Score 

(4) (5) (6) (7) 

Math Achievement 

Preschool Treatment 


Head Start Treatment Group -0.00 -0.00 0.11 

(0.07) (0.07) (0.16) 

Building Blocks Preschool Treatment Only 

Building Blocks Preschool Treatment with K and 
1 st Grade PD Extension 

0.17* 0.13+ 0.12 0.18* 

(0.08) (0.08) (0.08) (0.08) 
NI NI NI 0.31** 

(0.07) 

Instructional Quality 


Total advanced literacy activities in K and G1 0.02 0.04 

combined (times per month; standardized) 77^ 

Total basic literacy activities in K and G1 -0.12 -0.04 

combined (times per month; standardized) (0 ^ 1 ) 

COEMET (Good Instruction) 

Number of Math Activities 

0.03 0.03 

(0.04) (0.06) 

0.16** 0.22** 
(0.04) (0.08) 

Treatment Interactions 


Head Start Treat * Advanced literacy activities -0.04 

(0.25) 

Head Start Treat * Basic literacy activities -0.14 

Building Blocks Treat * COEMET 

Building Blocks Treat * Number of Math 

Activities 

Controls x x x 

Fixed effects for Random Assignment Block x x x 

0.02 

(0.09) 

-0.09 

(0.09) 

X X X X 

X X X X 

Observations 1056 1056 1056 

**p< 0 i ; * p < .05, + p< . 10 HSIS Notes ' Standard errors 

, , clustered by unit of random 

NI= Grou P not lncluded ln model - assignment (in parentheses). 

Controls: Head Start entry 
assessment scores, gender, race, 
teen mom, parents education, 
immigration status, parent age, 
urbanicity, treatment 'no-show', 
treatment 'crossover', random 

564 561 561 884 

BB Notes. Standard errors clustered 
at the preschool teacher level (in 
parentheses). Controls: preschool 
entry math score, gender, race, age at 
preschool entry, mother's education 
level, free/reduced price lunch status, 
whether limited English proficient, 
random assignment block. 


assignment block. 
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Table Appendix A. Coding scheme for instructional quality of literacy activities in the Head Start Impact Study 


Kindergarten literacy activities 

First grade literacy activities 

Listen to stories with no print 

basic 

Activity related to book 

basic 

Show child how to read a book 

basic 

Write letters of alphabet 

basic 

Write own name 

basic 

Learn names of letters 

basic 

Teach directional words like over and up 

basic 

Have children tell you a story 

basic 

Write letters of the alphabet 

basic 

Practice sounds letters make 

basic 

Learn the names of letters 

basic 

Listen to stories w. print 

basic 



Read books chosen by child 

basic 



Read text w controlled vocab 

basic 



Read text w strong phonemic pattern 

basic 



Read patterned or predictable text 

basic 



Hear storytellers 

basic 

Discuss new words 

advanced 

Language activities in mixed achievement groups 

advanced 

Have children tell you a story 

advanced 

Discuss new words 

advanced 

Practice the sounds that letters make 

advanced 

Read aloud 

advanced 

Listen to stories with print 

advanced 

Read silently 

advanced 

Rhyming words and families 

advanced 

Work in reading workbook 

advanced 



Write words from dictation 

advanced 



Use invented spellings 

advanced 



Read thematic text 

advanced 



Compose stories or reports 

advanced 



Publish child's writing 

advanced 



Perform plays/skits 

advanced 



Write stories in journal 

advanced 
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